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DESIGN STRATEGIES FOR FILE SYSTEMS'- 



Abstract 



This thesis describes a methodology for the analysis and 
synthesis of modern general purpose file systems. The two 
basic concepts developed are (1) establishment of a uniform 
representation of a file's structure in the form of virtual 
memory or segmentation and (2) determination of a hierarchy 
of logical transof rmat ions within a file system. These con- 
cepts are used together to form a strictly hierarchical or- 
ganization (after Dijkstra) such that each transformation 
can be described as a function of its lower neighboring 
transformation. In a sense, the complex file system is 
built up by the composition of simple functional transfor- 
mations. To illustrate the sepcifics of the design process, 
a file system is synthesized for an environment including a 
mul ti -computer network, structured file directories, and re- 
movable volumes. 



"This report reproduces a thesis of the same title sub- 
mitted to the Alfred P. Sloan School of Management and 
the Department of Electrical Engineering, Massachusetts 
Institute of Technology, in partial fulfillment of the 
requirements for the degree of Master of Science, June 
1969. 
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EVOLUTIOH or FILE SYSTEMS 



CHAPTER OHE 



Introduction 



1 volution of File Syste ms 

The evolution of general purpose file systeis parallels 
very closely the evolution of operating systems. This is net 
surprising since the concept of file systems grew cut cf the 
embryonic input-output control {IOC) functions of early 
operating systems and now represents the most significant 
component of most modern operating systems. 

There has been very little attention formally directed 

to the specific problem of analyzing operating systems. In 

1967, Saul Bosen collected together material for a beck, 

"Programming Systems and Languages»'<Rosen 67>, which was to 

be a distinctive selection of previously published and 

unpublished reports describing the most iapcrtant 

programming languages and discussing many of the test 

important operating system concepts. He was forced to 

conclude: 

"The paper on Operating Systems was prepared for 
presentation at the University of BichigaE 
Engineering Summer Conference, June 18-29, 1962. 
It has had fairly wide circulation as Hand Report 
P-258Q. The material covered has been of vital 
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iaportance in the developnent of the "classical" 
operating system, yet it is difficult to find an 
adequate treatnent outside of very long and 
usually dry systea aanuals. George Healy was one 
of the few working experts in the field who took 
the tile to write down soae of the basic 
principles of operating systeas and also cf 
asseably systeas.** 

Mr. Rosen's observations iaply that very little 
attention has been expended in the atteapt to generalize the 
functions of operating systeas. File systeas have also been 
severely neglected. 

In the early years of coaputing (roughly 1952-1962), 
prograaaers slowly aoved away froa the practice of 
approaching a bare aachine with card decks and sharpened 
pencils, fighting with the console for more or less extended 
periods of tiae, and leaving triumphantly with final results 
or in defeat with a ream of aachine diiBp<fiosin 69>. 
Operating systems have evolved, not so auch as a blessing, 
but as a practical necessity. As coaputers becaae faster a ad 
aore complex, it was no longer possible for an individual 
prograamer to be an expert in every phase of the prograiaing 
and aachine usage; he now must rely on the operating staff 
and systea prograaaers to provide the necessities of life. 

These operating systems were often ill-designed and 
usually specialized around a single goal. One of the first 
trulj successful operating systems was FMS (FORTfiAH Monitor 
System) for the IBM 709/7090/7094 family. Its name implies 
its specialization. As a result a large number of operating 
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systens appeared, each with its own operating procedure and 
specialization. These systeas were typically very closely 
tied to a prograaning language (e.g. FOBTRAN, COBOI., 
Assesbler) . 

Input Output Control Systeas (IOCS) eaerged as a part 
of the Operating Systea based on the siaple obserTation that 
all prograas per for a soae aaount of input and/or output. 
Therefore, rather than requiring each prograaaer to write a 
new set of input/output routines for each prograa, a cciaon 
and sufficiently flexible collection of routines were 
supplied with the Operating Systea. This situation becaae 
especially critical as coaputer 1/0 capabilities were 
extended to include high-speed, buffered, asynchronous 
channels which required coaplex prograa logic to efficiently 
perfora input- out put. 

Froa the crude beginnings of IOCS, file systeas 
followed a logical, though often slow, evolution. Once all 
physical input/output functions were localized in the IOCS, 
aany generalizations becaae possible. Usually, there is so 
iaportant difference aaong the aany tape drives available at 
an installation, so that any arbitrary tape unit aay be used 
for input or output to a prograa. Furtheraore, later runs at 
the saae or different installations need not use the saae 
unit as long as unique correspondences can be aaintained. lit 
first it was considered that the best practice in handling 
the choice of input-output units by the object prograa was 
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to iBClude unit assignaents as an assembly paraaeter cr to 
read in unit assignaents as data and initialize the program 
appropriately. This practice sorted well when it was 
followed, which was seldom. lith the advent of the 
near-uni Ttersal use of IOCS, a more foolproof and flexible 
manner of operating was to establish the correspondences as 
part of the IOCS, The object prograas dealt strictly in 
symbolic unit assignments. 

Since the object programs no longer interacted directly 
with the I/O units nor were even aware of unit assignments, 
additional degrees of freedom became available to the 
operating system, providing a more efficient and convenient 
environment. For example, the system could determine unit 
assignments automatically and dynamically, based upon 
complex criteria such as availability and performance (e.g. 
I/O interference, buffering, etc.). The actual technique of 
I/O (unbuffered, single-buffered, double-buffered, etc) 
could be removed from programmer concern. 

The proliferation of I/O device types, such as 
low-speed, medium-speed, high-speed and hyper-tapes, as well 
as drums and disks of all shapes and sizes, resulted in the 
expansion of IOCS to include capabilities that are now 
called data management or file system facilities. The basic 
notion exploited is that just as the programmer had little 
concern as to what tapes were to be used, he really does not 
care what device is used nor what method of I/C is employed 
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within broad logical ccnsttaints. For example, if a 
prograBBer wishes logically to treat his I/O data as 80 
column cards, the file system could physically utilize 
unit-record eqaipaent, tapes, disks, drums, data cells, or a 
host of other devices in various manners logically to 
simulate the effect of input-output using 80 column cards. 

This trend became irreversible with the advent of 
multi-tasking operating systems, since the availability of 
devices was continuously and dynamically changing. In such 
an environment, it becomes impractical and probably 
impossible to designate specific I/O units statically and 
arbitrarily in the program. 

The importance of these data management and file 
systems cannot be overly emphasized. Just as the assumption 
that programs perform input-output was a basic fact, it 
appears that the number and flexibility of I/O facilities 
demanded by programs are continuously increasing. 

K major factor in the rapid growth of file systems is 
the introduction of low cost, high capacity, high-speed, 
direct access devices such as disks, drums, and data cells, 
A description of direct access devices would emphasize the 
fact tkat they have two degrees of freedom rather than cnly 
one as with tape-like devices. Since these devices can be 
used for both sequential and direct access applications, the 
total amount of usage increases. Of course, the extra 
degrees of freedom necessitate more complex I/O routines and 
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furtfcer tighten the reliance on file s^stens tc petfora 
these functions. 

Direct access devices are usually as flexible as or 
■ore flexible than tape devices. Card-iaage or printer-iaage 
fixed record data types can be handled as nell as 
variable -length or structured data forns. Although these 
capabilities could be performed by the object prcgrai, the 
vast Kjority of these functions have been subsuied as 
by-products of the file systen. 

The second najor factor contributing to the rising 
inportance of file systens, as in early operating systeas, 
was necessity. This tiae it was due to the "inf creation 
explosion'*. As the nuaber of users, uses, and sophistication 
of use increased, the aaount of inf or nation in the fcras of 
prograas and data rose correspondingly. It was no Icnger 
convenient nor usually physically possible to haul the 
required boxes of prograas and data to and froa the aachine. 
This inforaatiofi was converted and aaintained in a aore 
compact but directly aachine processible fora, such as 
aagnetic tape or disk pack. Not only vere the individual 
prograas and data collections large, but the total auaber of 
distinct and unique files (i.e. prograas and data 
collections) was very large. It is not aoccaaon for a single 
progranaer to have to use froa 10 to 100 separate prograas 
and a roughly equivalent nuaber of data collections. This 
situation became especially acute with the increased use of 
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online systeas. & user at a renote teletype terainal coald 

not be expected to re-type and enter all his prograas and 

data from the tecoinal. They nast te peraanently saintained 

and stored at the central coaputer facility, altbcugh 

accessable and alterable under reaote terainal ccntrol. 

Quite obviously, it would be uneconoaic and unsanageable to 

store each unigne file on a separate tape or disk pack. 

Robert Bosin highlights these developaents in his recent 

survey of supervisor and aonitor systeas<RosiB 69>: 

**& file systea is especially necessary in any 
system which purports to provide realistic 
tiae-sharing. However, the advantages of this 
facility cannot be overlooked in a acre 
conventional enviroaaent". 

Thus, people were faced with the problem of using the 

I/O devices to store thouirands of peraanent files in 

addition to the traditional ase for input, output and 

"scratch" storage. Direct access devices provide the 

capability of storing hundreds or thousands of unique files 

and accessing thea in any order conveniently. This type cf 

direct access device usage results in aany side effects. The 

first problea, of course, involves a coaplex storage 

organization facility to locate "eapty" space on the device 

and a directory- like aechanisa to keep track of the 

individual files. Hany other facilities are usually 

required, such as a security systea to prevent unauthorized 

access to restricted files, and procedures to recover froa 

hardware or software failures. Of coarse, each installation 



8 I. IHTBODOCTION 

or group develops additional elegant file system 
capabilities to leet special reguireraents or to provide 
extensive flexibility. 

For the same reasons that prograaners utilize and rely 
on the file system, the operating system uses the facilities 
of the file system. For example, user identification (e.g. 
passwords, account numbers, etc.), accounting and charge 
information as well as system self-measurement data must be 
maintained dynamically using the facilities of the file 
system. The previously mentioned directories of "empty" 
space on direct access devices and the symbolic file 
directory and access control information are usually handled 
as system files. The operating system uses the file system 
capabilities to store the various processing programs (e.g. 
FORTH AS, COBOL, Assemblers, etc.) as well as many 
infrequently used supervisor routines. Furthermore, advanced 
operating systems perform "spooling", roll-in/roll-out, and 
paging in conjunction with the file system. It is not hard 
to realize that the file system is usually the most 
important component of an operating system in terms of the 
manpower required to develop and implement, and the amount 
of instrcKTtions and space used by the file system. 

Whereas the early operating systems along with their 
rudimentary file systems revolved around the need to support 
miscellaneous I/O functions for programming languages, 
modern file systems are at the very center of the operating 
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system. The supervisor, programming systems, and cbject 
prcgraffis are totally dependent on the file system. 
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ScoEg and Purpose 

The developnient of file systeas has suffered frci lany 
of the sane problens as that of programBing languages. 
Probably the single nest important problem was the excessive 
concern with efficiency. Of course efficiency is important, 
but in most current-day programming situations other 
factors, such as productivity and flexibility, are finally 
receiving their long-deserved attention. The question of 
efficiency can be put into proper perspective froi recent 
studies of real programming groups, where it has been found 
that the "best" programmer was up to 15 times more 
"efficient" than the least proficient programmer. It is not 
the function of this paper to get deeply involved in 
programming language controversies, but to illustrate the 
trends and changing attitudes. For example, if the original 
designers of FOBTH&N had not felt that its acceptance 
depended on the utmost attention to efficiency and, 
therefore, had not defined the language in terms cf the 
hardware capabilities of a specific machine, IBM 7C4, it is 
possible that the evolution of languages such as FOBTfiAN-IV, 
COBOL, &LGOL, and PL/I and generalized compiler techniques 
might have proceeded in a more organized fashion. The entire 
field of generalized approaches tc programming languages and 
compiler techniques has only recently emerged as a major 
factor in the computing profession. 
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File systeas have follonea a similar development. In 
the naae of "efficiency", each ne« file systeo was specially 
tailor€d to the original needs and environment of its 
intended use and very seldom could benefit from the 
experience or techniques of preceeding systems. As the 
demands on a given file system increased, new features and 
facilities were added, often with a "crowbar". Each cf these 
piece- leal file systems drove us further and further frcm an 
organized, generalized file system structure. 

Host literature in this area has appeared in one of two 
forms. The typical system manuals dearribe the "clever" 
techniques used to implement a specific file system, but 
provide very little assistance for comparisons with ether 
current systems or in the design of new file systems. The 
other type of reference deals with discussions of desirable 
characteristics for future file systems, usually emphasizing 
user facilities, but adds little insight into the problems 
of designing and implementing such a system. 

To a certain extent, generalized approaches have begun 
to evolve in "time-sharing" systems. In this paper such 
systems will be called conversational resource-sharing, 
since time is only one of many resources that are shared and 
it is the conversational or interactive nature cf these 
systems that is most easily distinguished from 
batch-oriented operating systems. 

These generalized file systems for conversational 
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resource-sharing operating systems developed both by design 
and necessity. In order to provide all the features required 
by user prograns and the supervisor, a flexible design nas 
essential. Furthermore, owing to the complexity of the 
environment and its dynamically changing aspects, it would 
be impossible to devise an "optimally efficient" strategy. 
The implementers were thus forced to abandon any attempt to 
make the system acre efficient and were free to develop a 
flexible system with a clear conscience. 

The goals of flexibility and efficiency need not be 
contradictory. In any multi-tasking system, which includes 
most modern, non-conversational, batch-oriented operating 
systems as well as conversational systems, I/C operations 
can be performed asynchronously by channels, and the central 
processor time can be utilized by executing other tasks 
while I/O is in progress. In this environment file system 
efficiency ceases to be of paramount concern. Furtheracre, 
individual user attempts to optimize performance could 
result in unnecessary inefficiencies due to conflicts with 
other tasks, such as excessive I/O interference from 
overloading the channels. The file system, aware of the 
total requirements, could provide a strategy that results in 
a more harmonious arrangement, increasing system throughput 
far more than individual user optimization could. 

Even in single-task or application-oriented operating 
systems, there is definite value to an organized. 



SCOPE ADD PDBPOSE 13 

generalized file systea. For nost large, complex user 
programs as well as compilers and assemblers, the program 
action including precise file system reguiremects cannot be 
statically determined since it is a dynamic function of the 
input data supplied. Therefore, a dynamically flexible file 
system could often outperform a specialized, but inflexible, 
file system. 

It is the purpose of this paper to present a general 
file system design. It is extremely important to start with 
a flexible but precise model although this design will 
probably need to be modified and made more detailed for any 
specific implementation. This issue was highlighted by 
Robert Sappaport in his thesis "Implementing Hulti-Prccess 
Primitives in a Hultiplexed Computer Syst€m"<Rapp 6€> which 
describes the development of the Traffic Controller for the 
HIT Project MAC Hultics System: 

"After having found acceptable solutions for 
the problems at hand, one asks oneself why it took 
so long to arrive at these solutions and was there 
any way to have done it more quickly? One might 
further ask if the arrived-at solutions are in any 
sense optimum? 

After being involved in designing a large 
system involving the work of many people, one gets 
the feeling that such problems as were encountered 
here are bound to crop up. The development of any 
large systei can only remain manageable if 
distinct parts of the system remain modular and 
independent . 

Without a theory of computing systems to fall 
back on, designing such complex systems becomes an 
art, rather than a science, in which it is 
impossible to prove the degree to which working 
solutions to problems aire in any sense optisun 
solutions. In much the same way as authors write 
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books, large coiputer systems go through several 
drafts before they begin to take shape. la the 
absence of a theory one can only cope with the 
cotplexity of the situation by proceeding in an 
orderly fashion to first produce an initial 
working aodel of the desired system. This part of 
the work represents the aajor effort of the design 
and inplenentation project. Once having arrived at 
this benchaark, nany of the problens nay then be 
seen in a clearer light and revisions to the 
working aodel are iapleaented auch aore quickly 
than were the original aodules. As to the 
development of a theory, one gets the inpression 
that it will be a long tiae in coaing." 

Therefore, while we await THE general theory of 

computer science, the file systea aodel presented in this 

paper will hopefully serve the need for an "initial working 

model" from which "problems may be seen in a clearer light". 
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CHAPfEB TBO 

Motivation Behind File System Design 

There are two basic goals to be satisfied by the file 
system design. It is necessary to (1) establish a uniforn 
representation of a file's structure and <2) deter line the 
hierarchy of logical transforiations that occur in a file 
systeffl. W. B. Henry's recent paper on hierarchical data 
managesent syste»s<Henry 69> discusses similar notions cf 
separating logical and physical file control, but differs 
significantly froi the approaches presented in this report 
in many fundamental ways. It should be a useful reference to 
a reader interested in other current research in this area. 

Onif orm Be present at ion of File Structure 

A typical computer system is portrayed by Figure 2. 1. 
Such a configuration usually has a varied assortsent of 
secondary storage devices in addition to the pri»ary 
storage. Programs and data must be in primary storage in 
order to be executed or operated upon, respectively. 

It is generally true that if primary storage si2€ was 
limitless and very inexpensive, there would be no need for 
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secondary storage (possible exceptions may be backup 
requirements and transfer of data). In the framevork of this 
report, the file system will be defined as the software 
mechanism that extends the capacity of primary storage by 
handling and coordinating the transfer of information tc and 
from the secondary storage devices. This definiticn is 
somewhat more restrictive than other common interpretations 
which include as part of the file system the physical 
devices or the programs that operate upon the data. In this 
interpretation the file system merely stores and transfers 
information but does not operate upon it. 
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Figure 2. 1 
Physical Coaputer Config oration 
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Early file systems were usually designed tc operate 
with specific applicaticn programs. Since there are 
potentially a very large nunber of different secondary 
storage devices, ^any of which can be used in more than one 
way (e.g. sequential or random access, blocked or unblocked, 
etc.) each file system limited itself specifically to those 
devices and organizations that were appropriate for its 
interded application. Figure 2.2 depicts the relaticnshi ps 
between the applications, the devices, and the file systeias. 

This type of development produced chaotic situations. 
It is somewhat analogous to assembly language programming 
without any established standard calling sequences or 
communication conventions, which makes it difficult, if not 
impossible, to use arbitrary programs as sufcrcutines. In 
particular, it was quite common to find that data files 
produced by the payroll programs, using their private file 
system, could not be accessed by the file system used by the 
personnel programs, and vice versa. As a result, there was 
much duplication of effort and confusion in the development 
and use of these early file systems. 
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Early File Systeas 



(Analogous to Asseably Language Programing) 
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More recently, the coapnter aanafactarers and operating 
systcB designers realized that it is possible to select a 
snail set of coanon logical file organizations (or access 
■ethod^ that can satisfy the needs of aost application 
programs. Farthernore, these access aethods could be 
designed in a flexible aanaer to operate on a variety of 
different devices and device organizations. This provided 
the user with a logically device- independent interface vith 
the file systea. Figure 2.3 illustrates this structure. 

This approach can be coapared vith the eaergence cf 
Problea Oriented Languages^ such as COBOL for business 
applications and F08TRA8 for scientific applications. The 
access aethods file systeas suffered the saae shortcoaings 
as the prograaaing languages: (1) despite claias, they were 
not really device independent, (2) occasionally it was 
necessary to resort to asseably language to overcoae cr 
bypass a restriction, and (3) it vas not possible to 
inter-aix access aethods (analogy would be to isteraix 
FORTHAH and COBOL subroutines) . 
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Figure 2.3 
Access flethods File Systems 

{Analogous to Earl; Prograaaing Languages, 
Such as FOBTBAN and COBOL) 
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In order to overcome the weakness in the access sethcds 
approach, it is necessary to design a sing le unifcri file 
representation that can (1) be used for every application 
and (2) be device independent, this idealistic goal is 
analogous to the search for "The" universal programming 
language, for which PL/I is probably the most ambitious 
attempt to date. 

It is reasonable to expect that such a uniform 
representation will be so atomic or primitive in fcrm that 
it will be desirable tc construct more powerful specialized 
access methods for the convenience of the typical user. 
Since the access methods are built upon the uniform 
representation, it is much easier to modify or implement new 
access methods or, if necessary, operate at the atomic level 
to bypass the restrictions of the access methods. This 
approach pushes the logical/physical separation of file 
system structure much farther as indicated in Figure 2.4, 
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The rationale behind the selection of a particular 
uniform representation is not trivial. For exasfle, there 
are three broad classes of common uniform representations: 

1. Stream - every file is treated as a continuous 
sequential stream of information. It is possible 
to access only the current position in the stream 
or reposition to the beginning of the stream. This 
representation can be implemented conveniently on 
almost all secondary storage devices, although it 
does not provide the user with very powerful or 
efficient features for many applications. 

2. Direct-Access - every file is treated as an 
ordered collection of items. Each item is directly 
accessable by means of a unique identifier 
corresponding to its position in the ordering. 
This representation, nhich corresponds to primary 
storage organization, is more powerful than 
Stream, but is very difficult to itplesent on 
intrinsically serial devices, such as magnetic 
tape. 

3. Associative - every file is treated as an 
unordered collection of items, each item is 
directly accessible by means of an identifier that 
has been "associated" with the item. This is a 
very flexible representation. Unfortunately, 
except for a small class of sophisticated 
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secondary storage devices the iBpleoentaticn is 
very coaplex and inefficient, 
Irregardless of the specific uniforn representation 
chosen, the important concept is that all files can be 
viewed as being identical in structure independent of the 
particular physical device on which the file is recorded. 
This generalization is depicted in Figure 2.5, which should 
be compared with Figure 2.1. 
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Hierarchy of Logical Transfo raatio ps 

Although a precise description of a file system will 
not te presented until later sections, there are several 
general characteristics of most file systems. In 
particular, a user specifies his request, such as read or 
write, by designating a file and an element within the file. 
Host advanced file systems allow considerable flexibility in 
the mechanism used to specify a file, it is typically 
described by means of a symbolic file name. Furthermore, the 
element within the file is specified in terms of the logical 
representation of elements in the particular file system 
which may or may not correspond to a precise physical 
specification of hew and where the element is stored. For 
example, a typical request might be of the form: 

"Read item 23 from file ALPHA into location 1561*." 

Realizing that information oust usually be stored on 
devices in somewhat obscure ways, there must be some 
sequence of transformations required to convert the user's 
request into its fineil form that physically operates en the 
secondary storage device. Quite often the transf oraaticn is 
viewed as a single step but that is a gross 
oversimplification that hides the fundamental mechanisms in 
use. In Figure 2.6 the conversion process is illustrated in 
terms of a discrete sequence of logical transformations. 

Since the specifics of these transformations nay not be 
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obvioas until the nore detailed sections later, a siiple 
analogy is presented in Figure 2,6 that loosely parallels 
the file systen transf ornations. The analogy is only 
intended to provide soae insight into the rationale behind 
each stage of the transformation. 

The process starts from the user's request to "read 
item 23 from the file ALPHA into location 1564". The first 
step is to convert the symbolic file name into a unique 
numeric file identifier. In the analogy, this corresponds to 
looking up John Doe's identifier which is a social security 
in this illustration. The purpose for using an identifier is 
basically the same in both cases. It is usually more 
convenient to store information, manually or automatically, 
by means of a unique numeric "key" rather than a symbolic 
name which may, under certain circumstances, not even be 
unique (i.e. there may be more than one John Doe in which 
case other factors must be considered in order to uniquely 
identify the person under consideration). 

The file identifier can then be used to conveniently 
access all the information known about a file, this 
information collectively is known as the file's descriptor. 
In the analogy, this would correspond to requesting all 
information in the social security records of 030-34-1234, 

Now that everything is known about the file, it is 
necessary to consider the specific operation to be 
performed. Using the file descriptor, a sequence cf logical 
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I/O coiamands can be produced. These are called logical I/O 
commands because they do not consider the physical 
characteristics of the secondary storage device to be used. 
This is analogous to putting an address on a letter which is 
usually done without considering the physical destination 
nor the route to be taken. 

In order to complete the transformation, the logical 
I/O commands must be converted into the appropriate sequence 
of physical I/O commands. This conversion may be trivial or 
complei depending upon the peculiarities of the device and 
I/O interfaces to the devices. In the analogy this process 
is performed at the post office where the address is used to 
determine the physical routing needed to get the letter to 
its destination. 

The final step in the process is the physical transfer 
of information. This is usually performed by means of 
software/hardware interactions to activate the appropriate 
device and confirm the successful completion of the request. 
Of course, in the analogy this transfer is accomplished by 
the postman ("neither rain nor snow nor dark of night...") 
assisted by trucks, planes, trains and other automation. 
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CH&PIEB THBEE 

File Systen DeEign Model 

Basic Con cep ts Osed In File Systea Design 

Two concepts are basic to the general file system model 
to b€ introduced. These concepts have been described by the 
terms "hierarchical modularity" and "virtual aenory". They 
ifill be discussed briefly below. 

Hierarchical Modularity 

The term "modularity" means many different things to 
different people. In the context of this paper we will be 
concerned with an organization similar to that proposed by 
Dijkstra<Dij}ts 67><Dijks 68> and RandelKRand 68>. The 
important aspect of this organization is that all activities 
are divided into sequential processes, A hierarchical 
structure of these sequential processes results in a level 
or ring organization wherein each level only communicates 
with its immediately superior and inferior levels. 

The notions of "levels of abstraction" or "hierarchical 
modularity" can best be presented briefly by an example. 
Consider an aeronautical engineer using a matrix inversion 
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package to solve space flight probleas. At his level of 
abstraction, the coapater is viewed as a aatrix inverter 
that accepts the satrix and control information as input and 
provides the inverted natrix as output. The application 
prograaaer who wrote the matrix inversion package need not 
have had any knowledge of its intended usage {superior 
levels of abstraction) . He might view the computer as a 
"FORTBaN machine", for example, at his level of abstraction. 
He need not have any specific knowledge of the internal 
operation of the FOHTRAN system {inferior level of 
abstraction), but only of the way in which he can interact 
with it. Finally, the FORTRAH compiler implementer operates 
at a different (lower) level of abstraction. In the above 
example the interaction between the 3 levels of abstraction 
is static since after the matrix inversion program is 
completed, the engineer need not interact, even indirectly, 
with the applications programmer or compiler itplementer. In 
the form of hierarchical modularity used in the file sjstem 
design model, the multi-level interaction is ccntisual and 
basic to the file system operation. 

There are several advantages to such an modular 
organization. Possibly the most important is the logical 
completeness of each level. It is easier for the system 
designers and impleraenters to understand the functions and 
interactions of each level and thus the entire system. This 
is cften a very difficult problem in very complex file 
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Hierarchical Levels 
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systems with tens or hundreds of thousands of instructions 
and hundreds of inter-dependent routines. 

Another by-product of this structure is "debugging" 
assistance. For ezaaple, when an error occurs it can usually 
be localized at a level and identified easily. The coiplete 
verification (reliability checlcoat) of a file system is 
usually an impossible task since it would require tests 
using all possible data input and system requests occuring 
in each potential "system state". In order to construct a 
finite set of relevant tests, it is necessary to consider 
the internal structure of the mechanism to be tested. 
Therefore, an important goal is to design the internal 
structure so that at each level, the number of test cases is 
sufficiently small that they can all be tried without 
overlooking an important situation. In theory, level would 
be checked-out and verified, then level 1, level 2^ etc., 
each level being more powerful, but because of the 
abstractions introduced, the number of "special cases" 
remains within bounds. 

Virtual Heaory 

There are four very important and difficult file system 
objectives: (1) a flexible and versatile format, (2) as much 
of the mechanism as possible should be invisible, (3) a 
degree of machine and device independence, and (4) dynamic 
and automatic allocation of secondary storage. There have 
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been several techniques developed to satisfy these 
objectives in an organized Banner; the concept exploited in 
this generalized file systea has been called 

•»segiBentation«<D€nn 65> or "naaed virtual ■eBory"<Daley 68>. 
Onder this system each file is treated as an ordered 
sequence of addressable eleaents, vhere each eleient is 
noroally the sane size unit as the Bain storage, a byte or 
word. Therefore, each individual file has the form of a 
"virtual" core aesory, froa whence the naae of the technique 
came. The size of each file is allowed to be arbitrary and 
can dynaaically grow and shrink. There is no explicit data 
fornat associated with the file; the basic operations of 
the file system move a specified number of elements between 
designated addresses in "real" aeaory and the "virtual" 
memory of the file system. 

There are several reasons for choosing such a file 
concept. In some systems the similarity between files and 
main storage is used to establish a single mechanism that 
serves as both a file system for static data and program 
storage and a paging systea<Lett 68><Daley 68><Denn 68><Salt 
68> for dynaaic storage aanagement. "Virtual memory" 
provides a very flexible and versatile format. 8hen 
specific formatting is desired, it can be accomplished by 
the outermost file system level or by the user program. For 
example, if a file is to be treated as a collection of 
card- image records, it is merely necessary to establish a 
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routine to access 80 characters at a tiae starting at byte 
locations 0, 80, 160, ... . Al»ost all other possible 
formats can be realized by similar procedures. 

Except for the formatting modules, the entire file 
system mechanism, including allocations, buffering, and 
physical location, is completely hidden and invisible tc the 
user. This relates closely to the objective of device 
independence. In many file systems the user must specify 
which device should be used, its record size (if it is a 
hardware formatable device) , blocking and buffering factors, 
and sometimes even the physical addresses. Although the 
parameters and algorithms chosen might, in some sense, be 
optimal, many changes might fee necessary if the program is 
required to run with a different configuration or 
environment. This strategy does not prevent the user from 
providing additional information, such as how often the file 
will be used and in what manner. The important factor is 
that this information is not necessary and its significance 
is determined by the file system rather than the user. 

There are very serious questions of efficiency raised 
by this file system strategy. Most of these fears can be 
eased by the following considerations. First, if a file is 
to be used very seldom (as in program developsent) , 
efficiency is not of paramount importance; if, on the ether 
hand, it is for long-term use (as in a commercial production 
program) , the device-independence and flexibility for change 
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and upkeep will be very i»portant. Second , by relieTing the 
prograaaer of the cosplexities of the forsats, deirices, and 
allocations, he is able to utilize his energy ncre 
constructively and creatively to develop clever algorithns 
relating to the logical stroc taring of his problea rather 
than clever "tricks" to overco«e the shortcomings or 
peculiarities of the file systei. third, in vien of the 
complexity of current direct-access devices, it is quite 
possible that the file systen will be better able to 
coordinate the files than the average user atteipting to 
specify critical parameters. 
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0Z.§E2i£M Of yile System Des i gn Hod el 

The file systen design nodel to be presented in this 
paper can be viewed as a hierarchy of seven levels. In a 
specific ioplesentation certain levels aay be farther 
sub-divided or combined as required. A recent study of 
several oodern file systems, vhich will be published in a 
separate report, attempts to analyze tbe systeis in the 
framework of this basic model. In general all of the systems 
studied fit into the model, although certain levels in the 
model are occasionally reduced to trivial form or are 
incorporated into other parts of the operating system. 

The seven hierarchical levels are: 

1. Input/Output Control System <IOCS) 

2. Device Strategy Hodules (DSM) 

3. Allocation Strategy Modules |ASH) 

t. File Organization Strategy Hodules (FOSH) 

5. Basic File System (BFS) 

6. Logical File System (LFS) 

7. Access Methods and User Interface 

The hierarchical organization can be described from the 
"top" down or from the "bottom" up. The file system wculd 
ordinarily be implemented by starting at the lowest level, 
the Input/Output Control System, and working up. It appears 
more meaningful, however, to present the file system 
organization starting at the most abstract level, the access 
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routines, and reaoving the abstractions as the levels are 
"peeled away". 

In the following presentation the teras "file naee", 
"file identifier", and "file descriptor" will be introduced. 
Detailed explanations cannot be provided until later 
sections, the following analogy gay be used for the reader's 
assistance. A person's naae (file naae) , due to the soaewhat 
haphazard process of assignaent, is not necessarily unique 
or manageable for coaputer processing. A unigue identifier 
(file identifier) is usually assigned to each person, such 
as a Social Security naaber. This identifier can then be 
used to locate efficiently the inforaation (file descriptor) 
known about that person. 

Access Methods (AH) 

This level consists of the set of routines that 
super iapose a f oraat on tbe file. In general there will 
probably be routines to sianlate sequential fixed- length 
record files, seguential variable- length record files, and 
direct-access fixed- length record files, for exaaple. Many 
aore elaborate and specialized foraat routines, also called 
access aethods or data aanageaent, can be supplied as part 
of the file systea. Obviously, a user aay write his own 
access aethods to augaent this level. 
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Hierarchical Pile System 
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Logical Pile System (LPS) 

Routines above this level of abstraction associate a 
sjnbolic nane with a file. It is the function of the Lcgical 
Pile System to use the synbolic file naie to find the 
corresponding unique "file identifier". Below this level the 
syabolic file naae abstraction is elininated. 

Basic Pile System (BPS) 

The Basic File System must convert the file identifier 
into a file descriptor. In an abstract sense, the file 
descriptor provides all information needed to physically 
locate the file, such as the "length** and "location" of the 
file. The file descriptor is also used to verify access 
rights <read-only, write-only, etc.), check read/write 
interlocks, and set up system-wide data bases. The Basic 
Pile System performs many of the functions ordinarily 
associated with "opening" or "closing" a file, finally, 
based upon the file descriptor, the appropriate POSH for the 
file is selected. 

Pile Organization Strategy Rodules (POSH) 

Direct- access devices physically do not resemble a 
virtual memory, h file must be split into many separate 
physical records. Each record has a unique address 
associated with it. The Pile Organization Strategy Hcdule 
maps a logical virtual memory address into the ccrrespcnding 
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physical record address and offset within the record. 

To read or write a portion of a file, it is necessary 
for the POSM to translate the logically contiguous virtual 
nenory area into the correct collection of physical records 
or portion thereof. If necessary, new records are allocated 
by the iSH, The list of records to he physically processed 
is passed on to the appropriate DSH. 

Although not necessary, the FOSH is often designed to 
allocate "hidden" file buffers in order to ninimize 
redundant or unnecessary I/O. If the requested portion of 
virtual aenory is contained in a currently buffered reccrd, 
the data can be transferred to the designated user aain 
storage area without intervening I/O. Conversely output to 
the file nay be buffered. If a sufficiently large nuaber of 
buffer areas are allocated to a file, it is possible that 
all read and write reguests can be perforaed by aerely 
Boving data in and out of the buffers. Hhen a file is 
"closed", the buffers are eaptied by updating the physical 
records on the secondary storage device and released fcr use 
by other files. Buffers are only allocated to files that are 
actively in use (i.e. "open"). 

allocation Strategy nodules (ASH) 

The Allocation Strategy Bodules keep track of the 

available records on a device. They are responsible for 

allocating records for a file that is being created or 
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expanded, and deallocating records for a file that is being 
erased or truncated. The FOSM requests that a record be 
allocated when needed, the iSM actually selects the record. 

Quite frequently, the ASH functions are incorporated 
into either the FOSM or DSM. In this paper these functions 
will be kept as separate as possible by explicitly 
recognizing the separate ASM level. 

Device Strategy Modules (DSM) 

When a large portion of a file is to be read or 
written, many records must be processed. The Device Strategy 
Module considers the device characteristics such as latency 
and access time to produce an optimal I/O sequence frea the 
FOSM and ASM requests. 

Input/Output Control System (IOCS) 

The Input/Output Control System coordinates all 
physical I/O on the computer. Status of all outstanding I/O 
in process is maintained, new I/O requests are issued 
directly if the device and channel are available, otherwise 
the request is queued and automatically issued as seen as 
possible. Automatic error recovery is attempted when 
possible. Interrupts from devices and unrecoverable error 
conditions are directed to the appropriate routine. Alnost 
all nodern operating systems have an IOCS. 
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File Systems versus Data Hanagement Systems 

In the literature there is often confusion between 
systems as described above, which this paper calls "file 
systems" and systems which will be called "data aanageitent 
systems", such as DI^-KDixon 67>, GIM-KNel 67>, and 
TDMS<Blei 67>. The confusion is to te expected since both 
types of systems contain all of the functional levels 
described above. The systems differ primarily en the 
emphasis placed on certain levels. 

In general file systems, the file is considered the 
most important itett and emphasis is placed on the directory 
organization (Logical File System) and the lower 
hierarchical levels. It is expected that specialized access 
methods will be written by users or supplied with the system 
as needed. 

In TBOst data management systems, the individual data 
items are considered the most important aspect, therefore 
emphasis is placed on elaborate access methods with aiciiial 
emphasis on the lower levels of abstraction. Because of the 
heavy emphasis on a single level, data management systems 
tend tc appear less hierarchical than file systems since the 
lower levels are often absorbed into the access methods. 
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Access H€thods 

The virtual »emory interface provided by the logical 
File System allows for very flexible user applications and 
access nethods. In a PL/1-like notation, calls to the 
logical File System are of the fora: 

IFS_ Be ad/* rite (Filename, iddrl, Addr2, Huaber) ; 
where Addrl is the main storage address, Addr2 is the file 
virtual memory address, and Number is the number of elements 
to b€ moved. 

In this paper elements will be assumed to be 8-bit 
bytes. For example, a request to read TOO bytes from 
location 200 within the file named ilPHi into main storage 
location 1231 could be expressed: 

LFS_Bead (» ALPHA* , 1234, 200, 100); 

Sequential fixed-length records, sequential 

variable-length records, and direct-access fixed-length 
records are common access methods. All cf these 
organizations and many more can be realized using a file's 
virtual memory. Note that the records processed by the 
access methods are "software" records and have no relation 
to the physical /logical records processed by the FOSM and 
DSH. 
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Sequential and Direct-Access Fixed-Length Record Access 
Methods 

To simulate these access aethods, the file's virtual 
aemory is treated as a sequence of records of the desired 
length, 1. 

To access these records sequentially, a position 
counter, PC, is set aside that starts at and is 
increaented by I, after each read or write. The position 
counter therefore finds the location of the next sequential 
record. The routine could be srittea as: 

LFS_Read (Filename, location, PC, 1); 
PC = PC + L; 

To access these records by direct-access there is no 
need for a position counter since the desired record, r, can 
be found at location (r-1)*L in the file's virtual aeiory. 
This routine could be written as: 

LFS_Read (Filenaae, Location, (r-1)*l, I); 

Sequential Variable-Length Record Access Method 

The Sequential Variable-Length Record Access Method 
treats the file as an ordered sequence of records, each 
record aay be a different length. This method can be 
implemented by preceeding each record with a "hidden" length 
field. 

These records can be accessed using a variation cf the 
Sequential Fixed-Length scheme. For example: 
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LFS_Head (Filenaae, L, PC, U) ; /♦ Get 1 byte 

length ♦/ 

LFS_aeaa (Filenaae, Location, PC+4, 1); /* Get 
data */ 

PC = PC * L ♦ 4; /* Update position counter ♦/ 

Other access Methods 

The above examples were presented to illustrate the 
ease with which conventional access methods can be supported 
under this file system design. The real importance of the 
virtual memory concept is not its ability to provide 
traditional access methods, but the ease and flexibility 
with which problem-oriented access methods can be developed. 
The programmer is able to design access methods based en the 
needs of his problem rather than forcing his problem 
solution to be constrained by a small set of limited access 
methods. For example, llelson<Bel 65> discusses some flexible 
and complex file structures that can be used "as an adjunct 
to creativity". 

The power of a computer reaches its peak when it is 
capable of amplifying the creativity of the programmer. A 
system that restricts the programmer's ability to express 
his ideas provides him questionable service. 
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Ii2ai£3i lAlS. Systeg 

A user's progran references each file by neans of a 
unique symbolic naae. It is the function of the Logical File 
Systea to convert the syiabolic naae reference into its 
corresponding unique file identifier. The Logical File 
System performs the mapping using a "file directory 
organization". 

In the simplest case the file directory is entirely 
stored in main storage as a tuo-entry table. The two entries 
are the symbolic file name and its corresponding file 
identifier. A look-up routine is all that is needed to serve 
the function of the Logical File System. This approach is 
used by several file systems because of its simplicity and 
efficiency. Unfortunately, the number of files that are 
allowed in the file system is restricted by the amount of 
main storage available for the file directory. 

To remove the above limitation, many file systems keep 
the file directory on secondary storage. The file directory 
can he treated as a standard file if its file descriptor is 
always known. This allows the file directory to be 
processed, expanded, and truncated using the normal file 
system mechanisms. The Logical File System mapping still 
involves a table look-up, only this time the table is 
contained in a file's virtual memory rather than main 
storage. The calls tc the Basic File System are essentially 
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the saae as the calls to the Logical File Systea, only a 
file identifier is specified rather than a syabclic file 
name. 

k fe» of the adranced file systeas have introduced the 
concept of the hierarchical file directory. Fro a a siaple 
point of irxev, a file directory hierarchy reseables and 
serves a siailar purpose to a PL/1 data structure. In 
practice, certain files are classified as "directories" in 
addition to their noraal attributes. The earlier aodel of 
the Logical File Systea iaplied that there was only one 
directory file. This file contained the file identifiers for 
all the other files, called "data files". This has been 
extended to allow the base directory, often called the "root 
directory", to contain file identifers for directory files 
as well as data files. Each subsequent directory file can 
contain file identifers for other directory files as well as 
data files. 

Figure 3.8 illostcates a file directory hierarchy. The 
files a, B, C, and D are directory files, all the others are 
data files. The data files, as well as directory files, do 
not necessarily have unique syabolic naaes. There are 3 data 
files in Figure 3.8 naaed "X", as in PL/1 this aabiguity is 
solved by using qualified naaes such as "A.X", "4.B.D.X", 
and "i.C.X". 

The file directory hierarchy serves aany purposes in 
addition to providing flexible and versatile facilities for 
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Figure 3,8 
Hierarchical File Directory Example 



56 III. FILE SYSTEH DESIGH HODEL 

progra»«er usage. "Pile sharing" and "controlled access" 
anong users are very closely tied to the hierarchical 
directories. Certain of these features are discussed in the 
paper by Daley and Ifeanann<Daley 65>. A more detailed 
treatment of this topic »ill be presented in a subsequent 
paper by this author. 

The iipleaentation of the Logical File Systei for a 
file directory hierarchy is a siiple extension of the single 
directory technique. After finding the correct file 
identifier in the root directory, it is either the data file 
desired or, if a secondary directory file, is used in 
exactly the saae Banner as the root directory identifier to 
advance one nore level in the hierarchy. 
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8asi£ ILIS Systea 

As explained in the Oirerview section, a file is 
physically located on secondary storage as an ordered 
collection of distinct records. The information that 
describes a file's size, access rights, device address cr 
addresses, and the sapping algoritha oust be naintaiued by 
the file systes. 

In a siaple file systea this information can be 
incorporated into the file directory as long as there is a 
nnigue one-to-one napping of file name onto file. In a 
sophisticated file system with features such as (1) 
hierarchical file directory, (2) aliases that allow a single 
file to be referenced by different names, (3) links that 
allow a file to be referenced from various directories in 
the file hierarchy or from different users, and (U) 
removable or detachable "volumes" or devices, the unigue 
mapping cannot be guaranteed. 

To produce an unambiguous file system, the file 
directory information is divided into three parts, the file 
name, identifier and the descriptor. The file name 
directories are the mappings between a symbolic file name 
and the corresponding identifer. The precise locations of 
the file descriptors can differ for different 

implementations, but uniguely defined by the identifer. In 
fact, since the file descriptors usually need not be 
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searched, they need not be contiguous. Usually they are 
collected in either <1) a special system wide file, (2) a 
collection of files, each located on a separate device or 
voluae, or (3) hidden within the symbolic file name 
directories. 

Although it is usually not possible to keep the 
symbolic file directories in main storage, the number of 
files actively in use is sufficiently small that the 
corresponding file descriptors can be placed in a 
core-resident table called the Active File Directory or Open 
File Directory. 

It is the function of the Basic File System to use the 
unique file identifier to locate the file descriptor and 
place it in the Active File Directory onless it has already 
been "opened". The Basic File System also checks that the 
action requested upon the file such as read, write, cr 
delete does not violate the restrictions specified in the 
file descriptor. 

After verifying legal access to the file, the Basic 
File System passes control to the appropriate File 
Organization Strategy Module as specified in the file 
descriptor entry. 
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File Organization Strat egy Modules 

The priaary function of the File Organization Strategy 
Module is to map a file's virtual nenory address ontc a 
corresponding physical record nunbcr. There are at least 
three comaon physical file organization strategies: 
sequential, linked, and indexed. 

Sequential File Organization Strategy 

The Sequential File Organization Strategy is used by 
most of the older, simpler, and non-dynamic file systems. 
Dnder this technique logically consecutive records are 
physically consecutive. For example, if each record is 1000 
bytes long, virtual address 321ft would be located in the 
fourth logical record. If the first logical record (i.e., 
the one containing virtual address 0) is physical record 
120, the record containing virtual address 3214 would be 
physical record 123. 

There are two notable advantages claimed for this 
technique. Firstly, the mapping is very sisple and 
efficient. The only information needed is the fixed record 
size and the address of the first record. Secondly, if the 
file is to be processed in a sequential manner, the 
consecutive organization allows for minimizing device 
latency and access time. 

Although the first point is indisputable, the second 
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claioed advantage is open to question. If there is lore 
than one file on the sane device that is actively in use, as 
is ccmion in a aulti- tasking erviroaaent, then the device 
read/vrite positioning will be switching rapidly aaong the 
active files, defeating the assaaed sequential accessing. 

The aajor disadvantage of this sequential organization 
is that the aaxiaua size of the file aust be assuied 
statically before creating the file. By specifying too 
small a size, the task will be forced to terainate if acre 
space is needed. If too large a size is assuaed, as is 
coBBon, there is luch wasted space and fragaentaticn. 

This technique aay be reccaaended for single- tasking 
systeas with few peraanent files and very few files 
siaultaneotisly in use. It aight be useful for a large 
inforaation utility systea which is based on a large nuaber 
of independent, low cost, low usage, high capacity devices 
such as data cells where wasted space is sot a significant 
problea. 

Linked File Organization Strategy 

The Linked and Indexed File Organization Strategies 
allow for files to dynaaically grow and shrink. The linked 
technique was probably developed first since it is siipler 
and eaphasizes sequential characteristics which were 
priaarily used in early file systeas. 

The linked organization reqeires each record of a file 
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to specify the location of the next logical record, 
analogous to the »'links" on a chain. The file descriptor 
specifies only the location of the first record. It tells 
nothing about the locations of the other records. As the 
file grows, new records are dynaaically allocated and linked 
onto the file. 

For seguentially processed files, the linked technique 
provides a very siiple and efficient aechanisn. A few bytes 
are used in each record to record the link, and since record 
sizes are usually in the range of 1000 bytes the overhead is 
niniaal. Onfortunately, randoi or direct-access file usage 
poses serious problems. If, for exaaple, the last access 
was to a data area in logical record 5, a reference to an 
area in logical record 15 will require 9 intermediate I/O 
accesses to find the links before reaching the desired 
record. The Linked File Organiaation Strategy has been used 
satisfactorily on systeas where the vast aajority of files 
are accessed sequentially. 

Indexed File Organization Strategy 

The Indexed File Organization Strategy is a 
significant variation to the linked technique. Records are 
dynaaically allocated as needed, but rather than 
distributing the record addresses throughout the file as 
links, they are collected together as a table. The logical 
record nnaber is used as an "index" in the table to find the 
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corresponding physical record nanber. 

If files are lioited tc ssall or nediun sizes, the 
index table can be stored as part of the file descriptor. 
If files are allowed to be arbitrarily large, the index 
table Bust itself be treated as a file and is broken into 
separate records. In the forner case, sequential and randoa 
access processing proceed easily and efficiently. In the 
latter case, sequential processing is very efficient, except 
for interoittent accesses for the next portion of the index 
table. BandOB processing aay he very efficient if localized 
to a sittple index table block; in any case it will never 
exceed a saall number of intemediate accesses, usually one 
or two, for totally random processing. 

The Indexed File Organization Strategy has the 
advantage of allowing the concept of a '♦sparsely filled" 
file. If we assume that each physical record is 1000 bytes 
and each index (record number) is 4 bytes, then the index 
table for a file that is 250,000 bytes long would require 
250 indexes or 1000 bytes. By designating a special cede, 
such as 0, to indicate an index for a non-allocated record, 
a file can be created with specific contents at locations 
10,000, a0,000, and 247,000 but with unspecified contents 
elsewhere. By convention, unspecified contents are usually 
initialized as zero by the file system. The above sparse 
file would only require four physical records, three records 
for the specified portions of the file and one record for 
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the index table. As lore inforaation is written intc a 
sparse file, lore physical records will be allocated as 
needed. 

The indexed organization provides a siaple and 
efficient nay to use prograasing technigaes, such as "hash 
coding" or "randoa entry" tables, that reguire a large 
though sparse virtual neaory. 

Many of the aost recent file systeas have adopted 
technigues siailar to the Indexed Pile Organization. 
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Allocation Strategy Modules 

When the FOSH saps a valid urite request onto a logical 
record for which a physical record has not been allocated, 
the *SH is called to find an ayailable record for use. There 
are two comnon techniques used to keep track of available 
records. The first technique links all available records 
together. This lethod is often used in conjunction with a 
Linked File Organization Strategy Module. The second 
technique uses a "bit lap" for each device, k bit aap is a 
function which operates on a bit string and describes the 
relationship between a bit position and a physical record on 
the device. For exaiple a convenient bit nap night be: bit 
corresponds to physical record 0, bit 1 to physical record 
1, etc. If a bit is set to 0, the corresponding record is 
available for allocation, otherwise it has already been 
allocated to a file. The bit nap provides a very ccnpact 
representation of the allocation infornaticn. The 

allocation states of a device with a capacity of 8,0CC,000 
bytes divided into 8000 1000-byte records can be stored in a 
1000 byte bit nap. In a file systen with a large nunber of 
high-capacity direct-access devices, it aay be inpossible to 
keep all the bit naps in nain storage. The bit lap nay be 
subdivided into sections, such as a separate bit nap for 
each group of 800 records. Only one section of the bit nap 
for a device is kept in nain storage at a tine, the 
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reoaining sections are left stored on the device. 

Since seqaential processing is a very coaaon file 
usage, the ASH aay attempt to allocate records to take 
advantage of this fact. Of course, any specific File 
Organization Strategy Module and Device Strategy Module 
group are expected to be cooperative with the Allocation 
Strategy Modules to optinize overall perforiance. The 
precise nature cf oeaningful cooperation would be too 
detailed to discuss in this paper. 
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DSiice Strategy Modules 

In addition to the obvious "read" and "write" 
functions, direct access devices often require additional 
I/O cofflmands, such as "seek" and "search", for proper 
positioning. The FOSH and ASM deal only with the logical act 
of of reading and writing, fhey transfer a set cf requests 
to the DS« of the form: "read record 2t into location 5400, 
read record H9 into location 6400, and write record 27 fron 
location 9324". The DSM must translate these requests into 
the obscure I/O list foraat required for the particular 
device. 

Furtheriore, due to the device characteristics such as 
latency and access tiae, the order in which the requests are 
performed affects the total amount of time that the device 
is kept "busy". For example, if records 24 and 27 are 
"closer", in some sense, to each other than record 49, it 
might be more efficient to read record 24, write record 27, 
and then position to read record 49, 
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Ipp at/Cutput Control S ystea 

The Input/Output Control System coordinates all the 
physical I/O on the cosputer. On aost oodern coaputers there 
are coaplex interdependencies aaong the physically 
independent I/O devices, Usually this dependency occurs due 
to the dedicated nature of "selector" channels and device 
control units that can switch to any device but can only 
service one device at a tiae. For very high-speed devices, 
such as druas, the aain storage access tiae can be an 
important factor. If too aany siaultaneous aeaory requests 
occur, "overrun" can occur resulting in erroneous data 
transmission. The IOCS keeps track of the status of all 
devices, control units, and channels, Uhen an I/O operation 
is requested, the IOCS checks to insure a clear path tc the 
device through the channels and control units and that no 
I/O capacity limits will be exceeded. If it is not possible 
to issue the requested I/O operation, the IOCS stores the 
request on a queue. The I/O will be issued at a later tiae 
when all conditions are satisfied. Since the I/O 
interdependencies aay exist aaong all devices, every I/O 
operation whether for the file systea or dedicated special 
purpose device must be funnelled through the IOCS. 

Although most aodern I/O devices are very reliable, 
spurious errors do occur. Osually the retry or recovery 
procedure is very siaple, in such a case the IOCS will 
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atteapt corrective aeasures. 

The caller to the IOCS is ioforaed of the status of his 
I/O request, for example (1) successful coapletion, (2) 
unrecoverable error condition, or (3) asynchronous 
interrupt. 

The sophistication and scope of the IOCS depends upon 
the devices to be handled and the goals of the file systei 
and operating system. 
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CHAPTER FODB 
Hulti-Coapater Hetwork Environment 

BacXgroand 

k general file systea design ■odel aust, of coarse, be 
nodified and elaborated to satisfy the needs of any specific 
desired file system enyironment. To illustrate the 
refinement process, a aniqae file system design vill be 
presented for a malti-computer network. 

Halti-compater networks are becoming an increasingly 
important area of computer technology<Had 68>. There are 
several significant reasons behind the growth of 
multi- computer networks: 

1. To increase the power of a computer installation 
in a modular manner, especially if <a) it is not 
possible to acquire a larger processor, (b) 
reliability is important, or (c) there are 
real-time or time-sharing constraints. 

2. To serve the co-ordination requirements of a 
network of regional computer centers. 

3. To support the accessibility to a nation-wide data 
base. 
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An example of the environment to be considered for this 
paper can be illustrated in Figure 4,1, This type of 
multi-computer network has been in limited use for several 
years in many configurations. The IBK 7094/7044 

Direct-Coupled System<Bosen 69> was probably one cf the 
earliest practical examples of such an inter-ccnnected 
arrangement. 

There are several implicit constraints imposed upcn the 
multi-computer system illustrated in Figure 4.1: 

1. Independence of Central Processors. 

Each of the central processors operate independently 
such that there are no direct processor-to-processor 
data transfer nor signaling, and furthericre there 
is no "master" processor. 

2. Bon -shared Hemory. 

Each central processor has its own main storage 
unit. These units are not shared with nor accessed 
by another central processor. 

3. Inter- locked Device Controllers. 

The device controllers act as "traffic cops" to the 
actual I/O direct access devices. They control the 
traffic between a computer's I/O channel and a 
selected I/O device, h single device controller will 
only accept reguests from one channel at a time and 
will only select one I/O device (among those under 
its control) at a time. Once a device controller 
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Figure 4,1 
Exanple of Hulti-coBpater File System Network 



72 IV. MDLTI-COMPOTEB HETiCfiK EHVIBCSHINT 

connects a channel nith a device, the connection 
reaains intact until the channel releases the device 
or an I/O error occnrs. 

The environient described above, although well within 
the boundaries of current technology, has not been the 
subject of auch investigation. Such configurations are 
presently very expensive and, therefore, chosen only for 
very specialized situations. Even then there are only t¥0 or 
three processors and very specialized software and 
operational factors. A discussion of the CP-67A:hs liae 
Sharing Systei <IBH 68a><Sea 68> will serve to establish the 
relevance of the lulti-coaputer network environaent. 

The CP-67/CHS Tiae Sharing Systea uses the special 
hardware features of a single IBM Syste«/360 aodel 67 
processor augaented by software to produce an apparant 
environaent corresponding to the aulti-coaputer network 
illustrated in Figure *. 1, with aany independent central 
processors, device controllers, and direct access I/O 
devices. In practice a typical single processor 360/67 
configuration would prodoce the affect of about 30 active 
processors ("virtual" Systea/360 aodel 65 processors each 
with a 256,000 byte aeaory) and 50 active device 
controllers. Hore detailed descriptions of the CP-67/CHS 
Systea can be found in the References. In the traditional 
sense of tiae- sharing, each user of the CP-67/CHS Systea is 
provided with a "virtual" coaputer operated froa a siaulated 
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operator console (actually an augmented remote terminal) . 
Most importantly, each "virtual" computer (i.e. user) 
operates logically independently of all other "virtual" 
computers except for the specified inter-connected I/O 
devices and device ccntrcllers. 
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Probleas Arising In Hulti-Conpeter Set wor ks 

There are lany problens associated with the 
aulti-computer file systea netvork. Soae of these problems 
are tinigoe to this environnent. Other probleas have been 
solved in traditional file systeBS<Corb 62><Salt 65><Scie 
68 >, but the solutions require major revisions due to the 
peculiarities of the environment. The most significant 
problems are listed briefly belov. 

1. No shared memory. 

Usually file systems co-ordinate the status of the 
files and devices by using main storage accessable 
tables and data areas that describe file status, 
access rights, interlocks, and allocation. There is 
no such common communication area in main storage 
that can be accessed by all the independent 
processors, 

2. No inter-ccmputer communication. 
Hulti-computer configurations usually provide a 
mechanism for sending signals or data transfers 
between the separate processors. Bith this 
capability the non-shared memory problem cculd be 
solved by either (a) electing one processor tc be 
the "master" processor that coordinates the ether 
processors, or (b) supply all the processors with 
enough icfcrmation sDch that each processor knows 
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what all the other processors are doing. The coBcept 
of a "laster" processor opposes the intesded 
boBogeneous, independent processor assasption. The 
possibility of supplying status inforaation to all 
other processors, although reasonable for a three or 
four processor configuration, was not considered a 
feasible solution for a systea with hundreds of 
processors and devices and thousands of files. For 
these reasons, inter-coaputer coinuBication, 

although an available capability, was not included 
as a reguired capability of the lulti-computer 
environnent described above. 

3. Ho pre-arranged allocations. 

For saall specialized ■ulti-coaiputer file networks, 
each processor can be "assigned" a specific area of 
a device or set of devices that can be used tc write 
new files, all ether processors can only read fro« 
this area by convention. This prevents the danger of 
two independent processors writing files at the saoe 
place. Such an "arrangement" is not practical for a 
large, flexible aulti-coaputer file network since 
the static assignaent of secondary storage space 
does not take account of the dynaiic and 
unpredictable reguireaents of the independent 
processors. 

1. Extendable device and file allocation. 
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The number of devices and sizes of devices as well 
as the Busber and sizes of files are, within reason, 
unlinited. For exaaple, a specific aiount of 
secondary storage equivalent to 100,000 card iaages 
could be used to hold 10 files of 10,000 card each 
or 1,000 files of 100 cards each. This consideration 
discourages techniques that result in a strong 
efficiency or nain storage capacity dependency on 
the "size and shape" of the file systen. Of course, 
the magnitude of the file system size will affect 
the operation, but arbitrary restrictions such as 
"no more than 64 files on a device" would be 
discouraged unless essential. 
5, Removable volumes. 

It has become common to differentiate between the 
I/O mechanism used to record or read information, 
called a "device", and the physical medium on which 
the information is stored, called a "voluae". For 
most drums and many disk units, the device and 
volume are inseparable. But, for magnetic tape units 
and many of the smaller disk units the volume, 
magnetic tape reel and disk pack respectively, are 
removable. It is intended that the file system 
include files that are on unmounted volumes 
(disconnected from an I/O device) as well as mounted 
volumes. Therefore, a configuration that consists of 
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ten disk units aay have a file systea that 
enconpasses hundreds cf volumes, only ten of which 
nay be actively in nse at a time. Since removing and 
mounting a volume takes several minutes of manual 
effort, it will be assumed that the "working set" of 
volumes (volumes that contain files that are 
actively in use) remains static for reasonable 
periods of time and is less than or equal tc the 
number of devices available. The fact that volumes 
are removable and interchangeable (i.e. may be 
mounted en different devices at different times) 
does affect the organization of the file system, for 
example, a scheme that involved linking files 
together by means of pointers (chained addressing) 
could require mounting volumes just to continue the 
path of the chain even though little or no "logical" 
information was requested from files on that volume. 
In the worst Cdise, it might be necessary to mount 
and unmount all the volumes of the file system to 
locate a desired file. Such a situation should 
definitely be avoided if not totally eliminated by 
the file system • 
6. Structured file directories and file sharing. 

In a traditional file system, the mapping between 
the symbolic file name and the corresponding file 
was accomplished by means of a single Master File 
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Directory. For modern file systems with thousands of 

files scattered over hundreds of yolunes, it became 

desirable, if not necessary, to form grcuEings of 

files by means of Secondary File Directories<Daley 

65>. These groupings are often used by the system to 

associate users nith files they oun (User File 

Directories). This capability is also available to 

the user to arrange his files into further 

sub-groups (libraries) or into separate 

project-related groupings. Occasionally it becomes 

necessary for a file to be included in tac cr more 

groupings (e.g. accessible by more than one User 

File Directory) with potentially different access 

privileges (protection) associated with each 

grouping. Hany of these features that are relatively 

easy to implement in a traditional file system are 

complicated by the introduction of independent 

processors and removable volumes. 

7, Fail-safe operation. 

Beliable operation is a very important requirement 
of a general purpose file system. There are many 
known techniques for I/O error and systematic backup 
and salvage procedures that are applicable to this 
environment. The important problem associated with 
the multi-computer network is that potential error 
conditions exist that are not normally found in 
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traditional single conputer file systems. For a 
single computer system, a processor error (including 
unexpected processor disconnection, i.e. "turning 
off") is a rare occurrence. Such a situation is 
remedied by repairing whatever physical hardware is 
necessary and then running a special "salvager" 
progran tc bring the file system into a well-defined 
operational state. In the environment of a 
lulti-coBputer network, processors may be connected 
or disconnected at any time without any awareness by 
the other processors, To prevent any inconsistent 
file system operation by the other processors and 
eliminate the need for usually time-ccnsuning 
salvage techniques, it is necessary to keep the file 
system in a well-defined consistent state at all 
times. 
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ft File System Design 

The purpose of the remainder of this paper is tc apply 
the organization presented in the File System Design Bedel 
section to solve the problems associated with a 
mult i- computer file system network. Discussion of the Access 
Hethods and Input/Output Comtrol System will be omitted. 
This is necessitated for brevity and consideration of the 
facts that the Access Methods are highly application 
oriented, as discussed in a previous section, and that the 
Input/Output Control System is usually a basic and ccsmcn 
component of all Operating Systems, The principal 
contribution of this model lies in the structure of the five 
other levels. 

LSSlcal File S ystem 

To present the goals and reguirements of the Logical 
File System in a brief and demonstrative manner, an example 
will b€ used. The reader should refer to Figure 4.2 for the 
following discussion. It is important that the peculiarities 
of the example, such as the choice of file names (e, g, 
"FIII6" and "DIBi*") , not be confused with the general 
characteristics of the Logical File System, 

In Figure 4.2, there are 12 files illustrated. 
Associated with each file is an identifier of the form 
"VOLICB)". The usage of this identifier will not be 
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Figure 1,2 
Example of File Directory Structure (to LFS) 
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discussed until later, in the meanwhile notice that each 
file's identifier is unique. The 12 files are divided into 2 
types, directory files (i.e. 701,1(3), ?0L2(3), ?CI3|2), and 
V013(5)) , and data files (i.e. ¥0L1(2), V011<6), V011 (4) , 
7011(5), 7012(4), 70L2(2), 7013(4), and 70L3(3)). The 
distinction between directory files and data files is onlj a 
natter of usage, the Access Hethods jaj operate open a 
directory file in the sase lanner as a data file, 
furtheraore, all Icwer levels (e.g. Basic File Systea) treat 
all files as data files, this factor will be elaborated 
shortly. 

It is the stated function of the logical File Systea to 
■ap a file naoe reference into a unique file identifier. 
This napping is a function of the requested file naoe 
(symbolic file nane path) and a starting point (base 
directory) in the file directory structure. In Figure 4.2, 
thre€ exanple base directories are illustrated by 
associating 70L1(3) with user 1, 7012(3) with user 2, and 
70L3(2) with user 3. Therefore, user 1 references to the 
file nane FIIE2 yields the file 7011(4). 

A nore conplez exanple can be illustrated by 
considering the file 70L3(4). Oser 3 can refer to this file 
under the nane FILB8. Alternatively, it can be referenced by 
the nane Dia3, FILET. The file DIR3, which is associated with 
70L3 (5) fron user 3*s base directory, is interpreted as a 
lower level directory. Then fron file 7013(5), the file nane 
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PILE? is mapped into yOL3(i») as intended. The file VOL3{4) 
can be referenced from user 2*s base directory as DIB 3, FILE 8 
or DIB3.DIF3, FILE7, for example. From user 1*s base 
directory, the file VOL3(4) can be referenced as FILE3, 
DIR2.DIR3. FILES, DIfi2.DIR3. DIE3. FILE?, or even 

DIH2.DIR3.DIRll.DIH3.Die3.FILE?. 

Two important side affects of the base file directory 
and file name path facilities are that (1) a specific file 
nay be referenced by many different names, and (2) the same 
name may be used to reference many different files. 

The headings VOLOMl "V0L1", VOLDME "V0L2«, and VOLOHE 
"V0L3" are intended to indicate that the 12 files are 
scattered over 3 separately detachable volumes: VOLI 
(containing V0L1|2), VOLI (3), VOLI (4), VOLI (5), and 
V0L1<6)), V0L2 {containing V012(2), V0I2<3), and VCL2{a)), 
and VOL 3 (containing V0L3(2), V0I3(3), VCL3(i|), and 
V0L3(5)). If volute V0L2 were detached from the system, user 
1 could still reference VOLI (Q) as FILE4 and V0L3(U} as 
FILE3, but could not reference V0I3 (4) as DIB2. DIB3. FILES 
nor V0L1(5) as DIR2.DIB3. DIR3. FILES since the path would 
logically reguire passing through volume V0L2. Furthermore, 
user 3 is allowed to erase (i.e. remove from file system 
structure) the file V0L3 (4) under the name FILES, assuming 
appropriate protection priviledges, whether or not volume 
V0L1 is mounted in spite of user 1's reference to file 
VOL3(4) under the name FIL13. 
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The logical File Systei could be extreaely ccsplex if 
it had to specifically consider the physical addresses of 
▼ olaaes, the device characteristics, and the location of 
file directories en volaoes, in addition to its obvious 
requirement of searching file directories. These problems 
are eliminated by introducing the file identifier and the 
interface with the Basic File System, 

The Basic File System processes requests that specify a 
file in terms of a file identifier consisting of a volume 
name and index, such as (V0L3,i)}, rather than a file name. & 
sample call from the Logical File System to the Basic File 
System, in PL/I-like notation, is: 

CILL BFS_READ(V0LDHE,I8DEX,C0BE_A1>DB,FIIE_ADDB,CCUBT) ; 
where VOLUHE is the name of the volume containing the file, 
INDEX is the corresponding onique index of the file, 
COfiE_*DDB is the main storage address into which data is to 
be read, FIlE_iDDB is the file virtual memory address from 
which the data is to be read, and COONT is the number of 
bytes to be transmitted. Using these features, the hear t of 
the logical File System (ignoring opening and closing files, 
file access protection, illegal file names, etc. ) reduces to 
the fl/I-like code presented in Figure 1.3, It is assumed 
that tte file name has been broken down intc an array of 
path element names (e.g. if name is 0IB2.DIB3. FILES, then 
PATH(1)='DTB2», PiTH (2) = 'DIBS', PATH{3) =»FILE8 • , and 

PATH_LE!IGTH=3) , that B11SE_V0LDHE and BASE_IHDEX initially 
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specify the (VOLUME, INDEX) identifier of the base airectcry, 
and that each entry in a file directory is N bytes long and 
fornatted as indicated in the FIL1_ENT8I declaration. 

For efficiency, the nanes of all files that are 
actively in use (asoally a snail fraction of all files in 
the system) are kept ia aain storage in an Active Haae 
Directory (AMD). The AND is searched before accessing the 
file directories en secondary storage. Entries are deleted 
from the AND when the corresponding file is "closed" or 
♦•deleted". 

Of course, the handling of access (protection) rights, 
errors, and other responsibilities will sake the logical 
File Systea much aore coaplex, but it is important tc note 
that the design and iapleaentation of the Logical File 
Syst€a escapes all physical file organization and device 
characteristic considerations and coaplexities. 
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DECISBE 1 FItE_EllTBY, 

2 FIlEMAflE CHABACTEE (8), 
2 VOLOHE CHiBACTEB (8), 
2 INDEX FIXED BINARY, 



DO I = 1 TO PATH_LENGTH; 

DO J = HI N HHILE (FILE^ENTBI .FIIEHABI -•= PATH{I)); 

CALL BFS_BEAD(BASE_¥OL0HE,BASE_IBDEX,FIIE_E»TBX,J*H,N) ; 

END; 

BASE_VOLDHE = FILE_ENTBY. tOLDHE; 

BASE_INDEX = FILE_ENTHY.INDE X; 
END; 



Figure H,3 
Exaaple Proceaure to Perfoca Logical File Systea Search 
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Basic File Systeg 

The Basic File Systea aust convert the file identifier 
supplied froa the Logical File System into a file descriptor 
than can be processed by the File Organization Strategy 
Module. A file descriptor contains inforaaticn such as the 
volume naae, physical location of the file on the vcluae, 
and the length of the file. Every file aust have an 
associated file descriptor, but since the number of passive 
files (i.e. not actively in use) sight be very large, the 
file descriptors are maintained on secondary storage until 
needed (i.e. file is "opened"). In organizing the secondary 
storage maintenance of the file descriptors there are 
several important considerations: 

1. There oust be a anigue file descriptor for each 
file regardless cf hon often the file appears in 
file directories or what symbolic names are used. 
This is reguiced to maintain consistent 
interpretation of a file's status. 

2. The file descriptor information for a file must 
reside on the same volume as the file. This is 
reasonable since if either the file or its 
descriptor is not accessable at some time by the 
system (i.e. unmounted) the file cannot be used, 
this possibility is minimized by placing thea en the 
same voluse. 
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3. In the same manner that the logical File System 

Has simplified by using the facilities of the lower 

hierarchical level, the file descriptors should be 

maintained in a manner that allows the File 

Organization Strategy nodule to process them as 

normal files. 

These problems are solved by the use of the Volume File 

Dea;riptor Directory (VFDD) . There is a single VFDD for each 

volume, it contains the file descriptors for all files 

residing on the volume. The file descriptors are of fixed 

length and are located within the VFDD pcsiticnally 

according to the corresponding file identifier's index. In 

order to exploit the facilities provided by the File 

Organization Strategy Module, the VFDD can be processed by 

the lower levels as a normal file. It is assigned an unigue 

file identifier consisting of the volume name and an index 

of 1, in fact the file descriptor for a VFDD is stored (when 

not in use) as its own first entry. Figure 9.4 presents 

diagrammatically the logical file structure of Figure 4.2 

with the added detail of the Volume File Descriptor 

Directories and File Directory formats. 

For efficiency, the descriptor's of all files that are 
actively in use are stored in an Active File Directory 
lAFD). The iFD is searched before accessing the Volume File 
Descriptor Directory. 
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Figure 4,4 
Example of File Directory Structure (to BFS) 
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The File Organization Strategy Bodule processes 
requests that specify a file in terms of a file descriptor 
(the entry extracted from the VFDD) rather than a file Eaie 
or file identifier, A sample call from the Basic File System 
to the File Organization Strategy Hodule, in PL/I-like 
notation, is: 

CAll FOSH_BE AD (DESCBIPTOB,COBE_AEDH,FILE_ADDB, COUNT) ; 
wher€ COfiE_ADDR, FILE_ADDB, and COUNT have the same 
interpretation as discussed above. 

The primary function of the Basic File System reduces 
to the single request; 

CALL F0SK_HEAB(VFDD_DESCBIPT0R,DESCfiIPTOR,l5* (INDlX-1) ,H) ; 
where VFDD_DESCBIPTOfi is the descriptor of the VFDD 
associated with the volume name supplied by the Logical File 
System as part of the file identifier, INDEX is from the 
specified file identifier, H is the standard length of a 
VFDD entry, and DESCBIPTOB is the desired file descriptor. 

The Basic File System performs several other tasks, 
such as protection validation and maintenance of the 
core-resident Active File Directory that enables efficient 
association between a file's identifier and descriptor for 
files that are in use (i.e. "open"). But, as in the Logical 
File System, the domain of the Basic File Systea is 
sufficiently small and narrow that it remains a conceptually 
simple level in the hierarchy. 
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File Organization Strategy Hodules 

The logical File System and Basic File Systei are, to a 
great extent, application and device independent. The File 
Organization Strategy Hodules are usually the acst critical 
area of the file systen in terms of overall perfcraance, for 
this reason it is expected that more than one strategy may 
be used in a large systea. Only one strategy will be 
discussed in this section, the reader may refer to the 
papers listed in the References <Corb 62><!!ad 68b><Salt 
65><Scie 68> for other possible alternatives. 

The FOSM must map the logical file address ontc a 
physical record address cr hidden buffer based upon the 
supplied file descriptor information. In the simplest case, 
the sapping could be performed by including a two-part table 
in the file descriptor. The first part of each entry would 
indicate a contiguous range of virtual file addresses, the 
second part of each entry would designate the corresponding 
physical record address. It has been assumed, however, that 
all file descriptors have a specific length, whereas the 
mapping table is a function of the file's length and is 
potentially guite large. Therefore, it is not feasible to 
include the entire mapping table as part of the file 
descriptor. One of the most powerful file organization 
strategies utilizes file maps. Figure ^,5 illustrates such 
an arrangement. 
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Figure 4.5 
Exanple of File Organization Strategy 
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In this example it is assumed that each file is divided 
into 1000 byte physical records. A file can be in one of 
several states depending upon its current length. If the 
file's length is in the range 1 to 999 bytes, the file 
descriptor contains the address of the correspcnding 
physical record. If the file is between 1000 and a99,999 
bytes long, the file descriptor specifies the address cf a 
file nap located on secondary storage. Each entry of the 
file sap (assuaed to reguire 2 bytes) designates the 
physical address of a block of the file (blocks are ordered 
by virtual file addresses: 0-999, 1000-1999, 2000-2999, 
etc.). Furthermore, for files greater than 500,000 bytes, 
but less than 250,000,000 bytes, there are 2 levels cf file 
naps as illustrated. 

This strategy has several advantages. Under the worst 
conditions of randon access file processing only frca one to 
three I/O operations need to be perforaed. By utilizing 
several hidden buffers for blocks of the file as well as 
file maps, the number of I/O operations required for file 
accesses can be drastically reduced. 
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Allocation Strategy Modules 

The function of allocation and deallocation of blocks 
involves several separate factors. Before describing the 
iffipleiaentation of the mechanisms, it is wise to revieti the 
desired characteristics: 

1. A file is allowed to grow in size, the FOSM will 
request additional blocks from the ASM for the 
data portions of a file or its index tables, as 
needed. 

2. Common direct access devices contain from 8000 to 
32000 separately allocatable blocks, thus it is 
not feasible to store all allocation infcraaticn 
in main storage, 

3. Since twc independent processors may be writing 
new files on the same volaae at the same time, it 
is necessary to provide interlocks such that they 
do not accidently allocate the same block to more 
than one file, yet not require one processor to 
wait until the other processor finishes. 

These problems can be solved by use of a special Volume 
Allocation Table (VAT) on each volume. In this scheme, a 
volume must be subdivided into arbitrary contiguous areas. 
For direct access devices with movable read/write heads, 
each discrete position {known as a "cylinder") covers an 
area of about HC to 160 blocks. A cylinder is a reasonable 
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unit of sabdivision. For each cylinder on the voluse, there 
is a corresponding entry in the VAT. Each entry contains a 
"bit map" that indicates which blocks on that cylinder have 
not been allocated. For example, if a cylinder consists of 
HO blocks, the bit map in the corresponding VAT entry would 
be 40 bits long. If the first bit is a «0", the first block 
has not been allocated; if the bit is a "1", the block has 
already been allocated. Likewise for the second, third, and 
remaining bits. 

When the FOSK first requests allocation of a blcck en a 
voluae, the ASM selects a cylinder and requests that the DSH 
read the corresponding VAT entry into main storage. An 
available block, indicated by a "0" bit, is located and then 
marked as allocated. As long as the volume remains in use, 
the VAT entry will be kept in main storage and blocks will 
be allocated on that cylinder. When all the blocks on that 
cylinder have been allocated, the updated VAT entry is 
written out and a new cylinder selected. With this technigue 
the amount of main storage required for allocation 
information is kept to a miniaua (about UQ to 160 bits per 
volume), at the sane time the number of extra I/O operations 
is minimized <abcut one per UO to 160 blocks of allocation) . 

The problem of interlocking the independent processors 
still remains. As long as the processors are allocating 
blocks on different cylinders using separate VAT entries, 
they may both proceed uninterrupted. This condition can be 
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acconplished by utilizing a hardware feature known as "keyed 
records'" available on several conputers including the IBH 
SysteiB/360, Each of the VAT entries is a separate record 
consisting of a physical key area and a data area. The data 
area contains the allocation information described above. 
The key area is divided into two parts: the identification 
number of the processor currently allocating blocks on that 
cylinder and an indication if all blocks on that cylinder 
have been allocated. A VAT entry with a key of all zeroes 
would identify a cylinder that was not currently in use and 
had blocks available for allocation. 

There are I/O instructions that can be used by the DSH 
that will autosiatically search for a record with a specified 
key, such as zero. Since the device controller will not 
switch processors in the aidst of a continuous stream of I/O 
operations from a processor (i.e. "chained I/C commands") , 
it is possible to generate an uninterruptible sequence of 
I/O coamands that will {^) find an available cylinder by 
searching the VAT for a entry with a key of zero and (2) 
change the key to indicate the cylinder is in use. This thus 
solves the multi -processor allocation interlock prcblea. 
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2§li£6 Stratecji W cdules 

The Device Strategy Hodules convert "logical I/O 
requests" from the File 0rgani2ation Strategy Modules and 
Allocation Strategy Modules into actual computer I/C command 
sequences that are forwarded to the Input/Output Control 
System for execution. 

When a request to transfer a large portion of a file 
{10,000 bytes for example) is issued, it is unlikely that a 
significant amount of the needed blocks are in hidden 
buffers. It will, therefore, be necessary to request I/O 
transfer for several blocks (e.g. about 10 blocks if each 
block 1000 bytes long) , The EOSM will generate logical I/O 
requests of the form: "read block 227 into location 12930, 
read block 211 into location 13930, etc.« The DSH must 
consider the physical characteristics of the device such as 
rotational delay and "seek" position for movable heads. It 
then decides upon an optimal sequence to read the blocks and 
generate the necessary physical I/O command sequence 
including positioning commands. The Input/Output Ccntrol 
System actually issues the physical I/O request, error 
retry, and ether housekeeping as discussed earlier. The 
detailed strategy for choosing the optimal I/O sequence is, 
of course, very device dependent and will not be elaborated 
here . 
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J2ik® JE Consideration s 

The proceeding sections have highlighted the fraaevcrk 
of a file systea. There are, of course, nany other iaportant 
decisions to be aade in such a systen, such as the foraat 
and organization of tables, error conditions<Lock 68>, 
measurement and accounting mechanisms, etc. Cne of the 
subtle points will be discussed in this section. 

The Basic File System is intended to deal with files 
represented by unique identifiers. In the specific system 
presented, the identifier is designated as the tuple, 
< volume, index in 7FDD>, This representation resulted in a 
very efficient mechanism for accessing a file's descriftor 
that avoided much of the time-consuaing table Icck-up. 
Unfortunately, this representation is not temporally unique. 
It has been assumed that when a file is deleted, the VFDD 
index position used for that file's descriptor is available 
for os€ by new files that nay be created. This would not be 
a problem if all instances of the deleted file's identifier 
were removed from the system at the same time, but there aay 
be more than one path to the file due to links frcm ether 
symbolic file directories. The strategy used by the Basic 
File System did net provide any convenient means tc locate 
all references (i.e. links) to a specific file. Furthermore, 
even if such a mechanism existed, it would not solve the 
problem since the reference may exist in a file directory 
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that is located on a volume that is not physically ncunted 
or accessable by the system at the time of deleticn. 
Therefore, in such an environment, it is possible to have 
links in directories that identify files that have been 
deleted. The danger exists that the following sequence of 
events may occur: (1) a file is created and assigned 
identifier, <A1PHI,5>, (2) a link is made to that file, (3) 
the file is deleted by its creator, {4) a new file is 
created and coincidently assigned the identifier <ALPHft,5>, 
and (5) the link previously created is used not realizing 
that the intended file has been deleted and replaced by scue 
other arbitrary file! 

Fortunately, this dilemna is not irrevocable, there is 
a multitude of solutions. Two simple variations would be (1) 
never reuse VFDD entries but allow the file to continually 
grow but become "sparse" or (2) maintain count of the nuEber 
of links to a file and reuse the VFDD entry only when all 
links have been removed. A better solution can be formulated 
by attacking the original goal of generating truly unique 
file identifiers. The Hultics Operating System has sinilar 
requirements, it forms unigue identifiers by concatenating 
the central processor's unique ^rial number with the 
chronolog clock time with accuracy in the range of 
microseconds. A much simpler scheme can be incorporated into 
the file system by associating a separate counter with each 
volume. Whenever a new file is created on a volume and 
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assigned a VFDD entry, the value of the corresponding 
counter is incremented hj one. For the purpose of the file 
system, the tuple, <volume, counter valuO, is a unique 
identification of a file. 

The counter value, which aonotonically increases, 
cannot be efficiently used as a direct index into a finite 
size file descriptor directory. A minor modification to the 
Basic File Systea design can incorporate the ideas of the 
above discussion. The file identifier can be constructed 
from the triple, < volume, 7FDD index, counter value>. In 
this context the counter value will be called a "key", since 
its sole purpose is to verify that the accessed VFDD entry 
is correct by attempting to "unlock" the entry (i.e. 
comparing the key from the VFDD entry with the key from the 
symbolic file directory which was copied from the VFDD when 
the link was initially established) . 

The above problems are typical of the factors that must 
be considered by file systea designers. The general file 
system model will very seldom be a complete description of a 
specific implementation and it certainly will net replace 
the need for systems analysts, but it can save many months 
of the initial design! 
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CONCLUDISG COBHEHTS 

Tc a large extent file systeas are carrentlj developed 
and iaplemented in much the same Banner as early "horse-less 
carriages", that is, each totally unique and "hand-«ade« 
rather than "nass produced". Coapilers, such as FOBTBAS, 
were once developed in this prinative aanner; but due to 
careful analysis of operation (e.g., lexical, syntax, and 
senantic analysis, etc.), compilers are sufficiently well 
understood that certain software companies actually offer 
"do-it-yourself POBTHAN kits". Since modern file systeis 
often outweigh all ether operating system components such as 
compilers, loaders, and supervisors, in terms of programmer 
effort and number of instrnctions, it is important that a 
generally applicable methodology be found for file system 
development. 

This paper presents a modular approach to the design of 
general purpose file systems. Its scope is broad enough to 
encompass most present file systems of advanced design and 
file systems presently planned, yet basic enough to be 
applicable tc more modest file systems. 

The file system strategy presented is intended to serve 
two purposes: (1) to assist in the design of new file 
systems and (2) to provide a structure by which existing 
file systems may be analyzed and compared. 
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